Bidirectional Encoder Representations Research Articles

A deep neural network model for Chinese toponym matching with geographic pre-training model

ABSTRACT Multiple tasks within the field of geographical information retrieval and geographical information sciences necessitate toponym matching, which involves the challenge of aligning toponyms that share a common referent. The multiple string similarity approaches struggle when confronted with the complexities associated with unofficial and/or historical variants of identical toponyms. Also, current state-of-the-art approaches/tools to supervised machine learning rely on labeled samples, and they do not adequately address the intricacies of character replacements either from transliterations or historical shifts in linguistic and cultural norms. To address these issues, this paper proposes a novel matching approach that leverages a deep neural network model empowered by geographic language representation model, known as GeoBERT, which stands for geographic Bidirectional Encoder Representations from Transformers (BERT). This model harnesses the groundbreaking capabilities of the GeoBERT framework by extending a generalized Enhanced Sequential Inference Model architecture and integrating multiple features to enhance the accuracy and robustness of the toponym matching. We present a comprehensive evaluation of the proposed method’s performance using three extensive datasets. The findings clearly illustrate that our approach outperforms the individual similarity metrics used in previous studies.

Data-Driven Strategies for Complex System Forecasts: The Role of Textual Big Data and State-Space Transformers in Decision Support

In this research, an innovative state space-based Transformer model is proposed to address the challenges of complex system prediction tasks. By integrating state space theory, the model aims to enhance the capability to capture dynamic changes in complex data, thereby improving the accuracy and robustness of prediction tasks. Extensive experimental validations were conducted on three representative tasks, including legal case judgment, legal case translation, and financial data analysis to assess the performance and application potential of the model. The experimental results demonstrate significant performance improvements of the proposed model over traditional Transformer models and other advanced variants such as Bidirectional Encoder Representation from Transformers (BERT) and Finsformer across all evaluated tasks. Specifically, in the task of legal case judgment, the proposed model exhibited a precision of 0.93, a recall of 0.90, and an accuracy of 0.91, significantly surpassing the traditional Transformer model (with precision of 0.78, recall of 0.73, accuracy of 0.76) and performances of other comparative models. In the task of legal case translation, the precision of the proposed model reached 0.95, with a recall of 0.91 and an accuracy of 0.93, also outperforming other models. Likewise, in the task of financial data analysis, the proposed model also demonstrated excellent performance, with a precision of 0.94, recall of 0.90, and accuracy of 0.92. The state space-based Transformer model proposed not only theoretically expands the research boundaries of deep learning models in complex system prediction but also validates its efficiency and broad application prospects through experiments. These achievements provide new insights and directions for future research and development of deep learning models, especially in tasks requiring the understanding and prediction of complex system dynamics.

Hype or hope? Ketamine for the treatment of depression: results from the application of deep learning to Twitter posts from 2010 to 2023

ObjectiveTo investigate societal perceptions of ketamine’s use in depression therapy by analysing Twitter posts from January 1, 2010 to April 1, 2023.MethodsUsing Twitter as the social media platform of choice, and employing search terms based on (depression OR depressed OR depressive) AND (ketamine OR esketamine OR Spravato), we collected English-language tweets from January 1, 2010, to April 1, 2023. Using unsupervised machine learning and natural language processing (NLP) techniques, including Bidirectional Encoder Representations from Transformers (BERT) and BERTopic, the study identified prevalent topics surrounding public chatter around the use of ketamine in depression treatment. Manual thematic analyses further refined these topics into themes.ResultsOut of an initial dataset of 99,405 tweets, after removing duplicate tweets, re-tweets and tweets posted by organizations over Twitter, 18,899 unique tweets from presumably individual users were analysed. Analysis of temporal trends revealed a shift in public attitudes, particularly after the United States Food and Drug Administration (FDA)’s 2019 approval of ketamine for depression. Three major themes emerged: a changing regulatory landscape, cautious optimism, and personal experiences with the drug. There was an initial spike in discussions post-FDA approval in 2019. Thereafter, cautious optimism (Theme 2) decreased among the general public, with more personal accounts (Theme 3) highlighting the potential benefits for some treatment-resistant patients. Limitations of the study include Twitter’s inherent biases towards younger, English-speaking demographics.ConclusionIn summary, the public’s multifaceted perception leans towards a hopeful stance on ketamine’s therapeutic potential for depression.

Depression Intensity Identification using Transformer Ensemble Technique for the Resource-constrained Bengali Language

Depression is an ordinary mental health-related disorder that hampers people’s daily activities, and sometimes, it destroys an individual’s life. It is one of the major social issues at present. Since depressed people use various social networking sites for sharing their thoughts and feelings, many scholars have tried to identify depression texts in highly resourced languages like English; however, only a small quantity of papers are detected in the resource-constrained Bengali language. This paper focuses on developing a depression intensity detection system from Bengali text data. In this regard, this study experiments on a 2,596 sample-sized dataset with four levels of depression by utilizing five state-of-the-art transformer models, including multilingual Bidirectional Encoder Representations from Transformers, DistilmBERT, XLM-RoBERTa, Bangla-BERT-Base, and BanglaBERT, and suggests a new ensemble method called MaxOfAvgProb. This method goes beyond the performance of the previous work on the same dataset, scoring 63.47% F1-score and 62.90% accuracy. To increase the reliability of the proposed method, we utilize this approach on another available dataset with 4,897 entries. In this case, our recommended method also surpasses the performance of the existing work on the same dataset, with accuracy at 86.45% and F1-score at 86.35%. Identifying the intensity of depression, depressed people may get proper counseling or treatment from their respected guardians or psychologists according to the victims’ level of depression.

Differing Content and Language Based on Poster-Patient Relationships on the Chinese Social Media Platform Weibo: Text Classification, Sentiment Analysis, and Topic Modeling of Posts on Breast Cancer

Background Breast cancer affects the lives of not only those diagnosed but also the people around them. Many of those affected share their experiences on social media. However, these narratives may differ according to who the poster is and what their relationship with the patient is; a patient posting about their experiences may post different content from someone whose friends or family has breast cancer. Weibo is 1 of the most popular social media platforms in China, and breast cancer–related posts are frequently found there. Objective With the goal of understanding the different experiences of those affected by breast cancer in China, we aimed to explore how content and language used in relevant posts differ according to who the poster is and what their relationship with the patient is and whether there are differences in emotional expression and topic content if the patient is the poster themselves or a friend, family member, relative, or acquaintance. Methods We used Weibo as a resource to examine how posts differ according to the different poster-patient relationships. We collected a total of 10,322 relevant Weibo posts. Using a 2-step analysis method, we fine-tuned 2 Chinese Robustly Optimized Bidirectional Encoder Representations from Transformers (BERT) Pretraining Approach models on this data set with annotated poster-patient relationships. These models were lined in sequence, first a binary classifier (no_patient or patient) and then a multiclass classifier (post_user, family_members, friends_relatives, acquaintances, heard_relation), to classify poster-patient relationships. Next, we used the Linguistic Inquiry and Word Count lexicon to conduct sentiment analysis from 5 emotion categories (positive and negative emotions, anger, sadness, and anxiety), followed by topic modeling (BERTopic). Results Our binary model (F1-score=0.92) and multiclass model (F1-score=0.83) were largely able to classify poster-patient relationships accurately. Subsequent sentiment analysis showed significant differences in emotion categories across all poster-patient relationships. Notably, negative emotions and anger were higher for the “no_patient” class, but sadness and anxiety were higher for the “family_members” class. Focusing on the top 30 topics, we also noted that topics on fears and anger toward cancer were higher in the “no_patient” class, but topics on cancer treatment were higher in the “family_members” class. Conclusions Chinese users post different types of content, depending on the poster- poster-patient relationships. If the patient is family, posts are sadder and more anxious but also contain more content on treatments. However, if no patient is detected, posts show higher levels of anger. We think that these may stem from rants from posters, which may help with emotion regulation and gathering social support.

What do you think caused your ALS? An analysis of the CDC national amyotrophic lateral sclerosis patient registry qualitative risk factor data using artificial intelligence and qualitative methodology

Objective: Amyotrophic lateral sclerosis (ALS) is an incurable, progressive neurodegenerative disease with a significant health burden and poorly understood etiology. This analysis assessed the narrative responses from 3,061 participants in the Centers for Disease Control and Prevention’s National ALS Registry who answered the question, “What do you think caused your ALS?” Methods: Data analysis used qualitative methods and artificial intelligence (AI) using natural language processing (NLP), specifically, Bidirectional Encoder Representations from Transformers (BERT) to explore responses regarding participants’ perceptions of the cause of their disease. Results: Both qualitative and AI analysis methods revealed several, often aligned themes, which pointed to perceived causes including genetic, environmental, and military exposures. However, the qualitative analysis revealed detailed themes and subthemes, providing a more comprehensive understanding of participants’ perceptions. Although there were areas of alignment between AI and qualitative analysis, AI’s broader categories did not capture the nuances discovered using the more traditional, qualitative approach. The qualitative analysis also revealed that the potential causes of ALS were described within narratives that sometimes indicate self-blame and other maladaptive coping mechanisms. Conclusions: This analysis highlights the diverse range of factors that individuals with ALS consider as perceived causes for their disease. Understanding these perceptions can help clinicians to better support people living with ALS (PLWALS). The analysis highlights the benefits of using traditional qualitative methods to supplement or improve upon AI-based approaches. This rapidly evolving area of data science has the potential to remove barriers to accessing the rich narratives of people with lived experience.

Multi-class multi-label classification of social media texts for typhoon damage assessment: a two-stage model fully integrating the outputs of the hidden layers of BERT

ABSTRACT With the development of social media, it has become increasingly important to quickly and accurately identify social media texts related to disasters (e.g. typhoon) to aid in rescue and recovery efforts. Currently, multi-class classification and pre-trained language model Bidirectional Encoder Representations from Transformers (BERT) are widely used for text classification. However, most studies on typhoon damage classification are multi-class single-label, which contradicts to the reality that a social media text may correspond to multiple types of damage. Moreover, the outputs of the hidden layers of BERT are not fully utilized. This paper proposes a two-stage multi-class multi-label classification method for typhoon damage assessment by fully integrating the outputs of the hidden layers of BERT. In the first stage, sentence vectors are adopted to identify typhoon damage-related texts. In the second stage, word matrices are applied for multi-class multi-label classification to further classify the texts into five damage categories (i.e. transportation, public, electricity, forestry, and waterlogging). The two stages are trained end-to-end to identify typhoon damage from social media texts. Experiments on SinaWeibo texts during typhoon landfall in Chinese coastal regions demonstrate that the proposed method can effectively improve the accuracy of text classification and comprehensively assess typhoon damage.

Improved random forest classification model combined with C5.0 algorithm for vegetation feature analysis in non-agricultural environments

In response to the challenges posed by the high computational complexity and suboptimal classification performance of traditional random forest algorithms when dealing with high-dimensional and noisy non-agricultural vegetation satellite data, this paper proposes an enhanced random forest algorithm based on the C5.0 algorithm. The paper focuses on the Liaohe Plain, selecting two distinct non-agricultural landscape patterns in Shenbei New District and Changtu County as research objects. High-resolution satellite data from GF-2 serves as the experimental dataset. This paper introduces an ensemble feature method based on the bagging concept to improve the original random forest classification model. This method enhances the likelihood of selecting features beneficial to classifying positive class samples, avoiding excessive removal of useful features from negative samples. This approach ensures feature importance and model diversity. The C5.0 algorithm is then employed for feature selection, and the enhanced vegetation index (EVI) is utilized for vegetation coverage estimation. Results indicate that employing a multi-scale parameter selection tool, combined with limited field-measured data, facilitates the identification and classification of plant species in forest landscapes. The C5.0 algorithm effectively selects classification features, minimizing information redundancy. The established object-oriented random forest classification model achieves an impressive accuracy of 94.02% on the aerial imagery for forest classification dataset, with EVI-based vegetation coverage estimation demonstrating high accuracy. In experiments on the same test set, the proposed algorithm attains an average accuracy of 90.20%, outperforming common model algorithms such as bidirectional encoder representation from transformer, FastText, and convolutional neural network, which achieve average accuracies ranging from 84.41 to 88.33% in identifying non-agricultural artificial habitat vegetation features. The proposed algorithm exhibits a competitive edge compared to other algorithms. These research findings contribute scientific evidence for protecting agricultural ecosystems and restoring agricultural ecosystem biodiversity.

Oversampling effect in pretraining for bidirectional encoder representations from transformers (BERT) to localize medical BERT and enhance biomedical BERT

BackgroundPretraining large-scale neural language models on raw texts has made a significant contribution to improving transfer learning in natural language processing. With the introduction of transformer-based language models, such as bidirectional encoder representations from transformers (BERT), the performance of information extraction from free text has improved significantly in both the general and medical domains. However, it is difficult to train specific BERT models to perform well in domains for which few databases of a high quality and large size are publicly available. ObjectiveWe hypothesized that this problem could be addressed by oversampling a domain-specific corpus and using it for pretraining with a larger corpus in a balanced manner. In the present study, we verified our hypothesis by developing pretraining models using our method and evaluating their performance. MethodsOur proposed method was based on the simultaneous pretraining of models with knowledge from distinct domains after oversampling. We conducted three experiments in which we generated (1) English biomedical BERT from a small biomedical corpus, (2) Japanese medical BERT from a small medical corpus, and (3) enhanced biomedical BERT pretrained with complete PubMed abstracts in a balanced manner. We then compared their performance with those of conventional models. ResultsOur English BERT pretrained using both general and small medical domain corpora performed sufficiently well for practical use on the biomedical language understanding evaluation (BLUE) benchmark. Moreover, our proposed method was more effective than the conventional methods for each biomedical corpus of the same corpus size in the general domain. Our Japanese medical BERT outperformed the other BERT models built using a conventional method for almost all the medical tasks. The model demonstrated the same trend as that of the first experiment in English. Further, our enhanced biomedical BERT model, which was not pretrained on clinical notes, achieved superior clinical and biomedical scores on the BLUE benchmark with an increase of 0.3 points in the clinical score and 0.5 points in the biomedical score. These scores were above those of the models trained without our proposed method. ConclusionsWell-balanced pretraining using oversampling instances derived from a corpus appropriate for the target task allowed us to construct a high-performance BERT model.

Deteksi Cyberbullying Menggunakan BERT dan Bi-LSTM

Cyberbullying is a digital problem that is not a new phenomenon. This existed before the advent of social networks, and cyberbullying has a wide impact, including a person's mental and physiological conditions such as sadness, anxiety and depression. The main objective of this research is to develop an effective cyberbullying detection system using natural language processing techniques. The method used in this research includes the application of the BERT (Bi-Directional Encoder Representations from Transformers) and Bi-LSTM (Bi-Directional Long Short-Term Memory) models as a deep learning approach to analyze text and detect cyberbullying behavior. This approach allows the system to understand complex language contexts and capture patterns that traditional methods may find difficult to identify. Testing was carried out using a dataset that included various types of Indonesian language texts containing cyber bullying acts. The research results show that the combination of BERT and Bi-LSTM is able to provide superior detection performance with a high accuracy rate of 90% and the ability to identify variations of cyber bullying. This research makes a significant contribution to efforts to protect individuals from the negative impacts of cyber bullying through the development of a sophisticated and adaptive detection system.

Fusion inception and transformer network for continuous estimation of finger kinematics from surface electromyography

Decoding surface electromyography (sEMG) to recognize human movement intentions enables us to achieve stable, natural and consistent control in the field of human computer interaction (HCI). In this paper, we present a novel deep learning (DL) model, named fusion inception and transformer network (FIT), which effectively models both local and global information on sequence data by fully leveraging the capabilities of Inception and Transformer networks. In the publicly available Ninapro dataset, we selected surface EMG signals from six typical hand grasping maneuvers in 10 subjects for predicting the values of the 10 most important joint angles in the hand. Our model’s performance, assessed through Pearson’s correlation coefficient (PCC), root mean square error (RMSE), and R-squared (R2) metrics, was compared with temporal convolutional network (TCN), long short-term memory network (LSTM), and bidirectional encoder representation from transformers model (BERT). Additionally, we also calculate the training time and the inference time of the models. The results show that FIT is the most performant, with excellent estimation accuracy and low computational cost. Our model contributes to the development of HCI technology and has significant practical value.

Analysis of the retraining strategies for multi-label text message classification in call/contact center systems

Today, in many areas of technology, we can come across applications of various artificial intelligence methods. They usually involve models trained on some specific pool of learning data. Sometimes, however, the data analyzed by these solutions can change its nature over time. This usually results in a decrease in classification efficiency. In such a case, the use of techniques to retrain the originally trained reference models should be considered. One of the industries where the nature of data changes quite dynamically over time is the broadly defined call/contact center systems. An example of a module that is often found in this type of system and that, due to frequently changing marketing campaigns, requires the use of learning techniques is the automatic classification of text data. The paper describes the process of retraining the original reference models used in a multi-label text message classification method dedicated directly to call/contact center systems applications. In order to carry out the retraining process, Polish-language data from the actual archives of a large commercial contact center system and English-language data extracted from a publicly available database were used. The study was conducted for models based on artificial neural networks and bidirectional encoder representations from transformer type models. In addition, two different retraining strategies were studied, the results of which were compared with data obtained from the operation of reference models. As a result of the research work, an improvement of up to 5% in classification efficiency, as described by the metric Emotica was obtained, which means that proper integration of the retraining process brings tangible benefits to the solution tested in the article. Thus, it can also benefit the solutions used in business.

USE OF NATURAL LANGUAGE PROCESSING TECHNIQUES TO PREDICT PATIENT SELECTION FOR TOTAL HIP ARTHROPLASTY: RESULTS FROM THE AI TO REVOLUTIONISE THE PATIENT CARE PATHWAY IN HIP AND KNEE ARTHROPLASTY (ARCHERY) PROJECT

To examine whether Natural Language Processing (NLP) using a state-of-the-art clinically based Large Language Model (LLM) could predict patient selection for Total Hip Arthroplasty (THA), across a range of routinely available clinical text sources.Data pre-processing and analyses were conducted according to the Ai to Revolutionise the patient Care pathway in Hip and Knee arthroplasty (ARCHERY) project protocol (https://www.researchprotocols.org/2022/5/e37092/). Three types of deidentified Scottish regional clinical free text data were assessed: Referral letters, radiology reports and clinic letters. NLP algorithms were based on the GatorTron model, a Bidirectional Encoder Representations from Transformers (BERT) based LLM trained on 82 billion words of de-identified clinical text. Three specific inference tasks were performed: assessment of the base GatorTron model, assessment after model-fine tuning, and external validation.There were 3911, 1621 and 1503 patient text documents included from the sources of referral letters, radiology reports and clinic letters respectively. All letter sources displayed significant class imbalance, with only 15.8%, 24.9%, and 5.9% of patients linked to the respective text source documentation having undergone surgery. Untrained model performance was poor, with F1 scores (harmonic mean of precision and recall) of 0.02, 0.38 and 0.09 respectively. This did however improve with model training, with mean scores (range) of 0.39 (0.31–0.47), 0.57 (0.48–0.63) and 0.32 (0.28–0.39) across the 5 folds of cross-validation. Performance deteriorated on external validation across all three groups but remained highest for the radiology report cohort.Even with further training on a large cohort of routinely collected free-text data a clinical LLM fails to adequately perform clinical inference in NLP tasks regarding identification of those selected to undergo THA. This likely relates to the complexity and heterogeneity of free-text information and the way that patients are determined to be surgical candidates.

Named entity recognition model based on Multi‐BiLSTM and competition mechanism

AbstractA named entity recognition (NER) model based on multiple bidirectional long short‐term memory networks (Multi‐BiLSTM) and competition mechanism (CM) is proposed. This model includes three parts: word vectorization module, learning module and application module. In the word vectorization module, the word vectors containing semantic features are generated based on BERT (Bidirectional Encoder Representations from Transformers). In the learning module, the multiple small‐scale BiLSTM are used to extract the contextual features of the word vectors synchronously. In the application module, the competition mechanism is adopted to select the relatively better models obtained in the learning module for recognizing entities. The results show the proposed model can reduce the parameters in the learning process, and the NER performance of the proposed model is better than that of the BERT‐BiLSTM‐CRF model. For example, the F1 can be increased by 1.02% and 0.39% for CoNLL2003 and Chinese resume datasets respectively.

Building large-scale registries from unstructured clinical notes using a low-resource natural language processing pipeline

Building clinical registries is an important step in clinical research and improvement of patient care quality. Natural Language Processing (NLP) methods have shown promising results in extracting valuable information from unstructured clinical notes. However, the structure and nature of clinical notes are very different from regular text that state-of-the-art NLP models are trained and tested on, and they have their own set of challenges. In this study, we propose Sentence Extractor with Keywords (SE-K), an efficient and interpretable classification approach for extracting information from clinical notes and show that it outperforms more computationally expensive methods in text classification. Following the Institutional Review Board (IRB) approval, we used SE-K and two embedding based NLP approaches (Sentence Extractor with Embeddings (SE-E) and Bidirectional Encoder Representations from Transformers (BERT)) to develop comprehensive registry of anterior cruciate ligament surgeries from 20 years of unstructured clinical data at a multi-site tertiary-care regional children's hospital. The low-resource approach (SE-K) had better performance (average AUROC of 0.94 ± 0.04) than the embedding-based approaches (SE-E: 0.93 ± 0.04 and BERT: 0.87 ± 0.09) for out of sample validation, in addition to minimum performance drop between test and out-of-sample validation. Moreover, the SE-K approach was at least six times faster (on CPU) than SE-E (on CPU) and BERT (on GPU) and provides interpretability. Our proposed approach, SE-K, can be effectively used to extract relevant variables from clinic notes to build large-scale registries, with consistently better performance compared to the more resource-intensive approaches (e.g., BERT). Such approaches can facilitate information extraction from unstructured notes for registry building, quality improvement and adverse event monitoring.

DR-BERT: A protein language model to annotate disordered regions.

Despite their lack of a rigid structure, intrinsically disordered regions (IDRs) in proteins play important roles in cellular functions, including mediating protein-protein interactions. Therefore, it is important to computationally annotate IDRs with high accuracy. In this study, we present Disordered Region prediction using Bidirectional Encoder Representations from Transformers (DR-BERT), a compact protein language model. Unlike most popular tools, DR-BERT is pretrained on unannotated proteins and trained to predict IDRs without relying on explicit evolutionary or biophysical data. Despite this, DR-BERT demonstrates significant improvement over existing methods on the Critical Assessment of protein Intrinsic Disorder (CAID) evaluation datasetand outperforms competitors on two out of four test cases in the CAID 2 dataset, while maintaining competitiveness in the others. This performance is due to the information learned during pretraining and DR-BERT's ability to use contextual information.

Leveraging mobile NER for real-time capture of symptoms, diagnoses, and treatments from clinical dialogues

In the dynamic world of healthcare technology, efficiently and accurately extracting medical data from physician-patient conversations is vital. This paper presents a new approach in healthcare technology, employing Natural Language Processing (NLP) to identify and extract critical information from doctor-patient conversations on mobile devices. Unlike traditional methods that rely on Electronic Health Records, our novel application enables the extraction of symptoms, diagnoses, and treatments directly on a mobile device during medical consultations, significantly enhancing patient privacy. We managed to integrate both Bidirectional Encoder Representations from Transformers (BERT) models and optimized Large Language Models (LLMs) on a mobile device without compromising performance significantly. Our findings reveal that the BERT model attained an F1-score of 85.1%, while FLERT and its compressed variant DistilFLERT showed superior performance. The FLAN-T5 model outperformed all models we tested with scores up to 92.7%. These results highlight the efficacy of leveraging advanced NLP and LLM technologies in healthcare environments on a mobile device, offering a promising direction for accessible and efficient patient care.

Efficient screening of pharmacological broad-spectrum anti-cancer peptides utilizing advanced bidirectional Encoder representation from Transformers strategy

In the vanguard of oncological advancement, this investigation delineates the integration of deep learning paradigms to refine the screening process for Anticancer Peptides (ACPs), epitomizing a new frontier in broad-spectrum oncolytic therapeutics renowned for their targeted antitumor efficacy and specificity. Conventional methodologies for ACP identification are marred by prohibitive time and financial exigencies, representing a formidable impediment to the evolution of precision oncology. In response, our research heralds the development of a groundbreaking screening apparatus that marries Natural Language Processing (NLP) with the Pseudo Amino Acid Composition (PseAAC) technique, thereby inaugurating a comprehensive ACP compendium for the extraction of quintessential primary and secondary structural attributes. This innovative methodological approach is augmented by an optimized BERT model, meticulously calibrated for ACP detection, which conspicuously surpasses existing BERT variants and traditional machine learning algorithms in both accuracy and selectivity. Subjected to rigorous validation via five-fold cross-validation and external assessment, our model exhibited exemplary performance, boasting an average Area Under the Curve (AUC) of 0.9726 and an F1 score of 0.9385, with external validation further affirming its prowess (AUC of 0.9848 and F1 of 0.9371). These findings vividly underscore the method's unparalleled efficacy and prospective utility in the precise identification and prognostication of ACPs, significantly ameliorating the financial and temporal burdens traditionally associated with ACP research and development. Ergo, this pioneering screening paradigm promises to catalyze the discovery and clinical application of ACPs, constituting a seminal stride towards the realization of more efficacious and economically viable precision oncology interventions.

Two in One: A multi-task framework for politeness turn identification and phrase extraction in goal-oriented conversations

Goal-oriented dialogue systems are becoming pervasive in human lives. To facilitate task completion and human participation in a practical setting, such systems must have extensive technical knowledge and social understanding. Politeness is a socially desirable trait that plays a crucial role in task-oriented conversations for ensuring better user engagement and satisfaction. To this end, we propose a novel task of politeness analysis in goal-oriented dialogues. Politeness analysis consists of two sub-tasks: politeness turn identification and phrase extraction. Politeness turn identification is dependent on textual triggers denoting politeness or impoliteness. In this regard, we propose a Bidirectional Encoder Representations from Transformers-Directional Graph Convolutional Network (BERT-DGCN) based multi-task learning approach that performs turn identification and phrase extraction tasks in a unified framework. Our proposed approach employs BERT for encoding input turns and DGCN for encoding syntactic information, in which dependency among words is incorporated into DGCN to improve its capability to represent input utterances and benefit politeness analysis task accordingly. Our proposed model classifies each turn of a conversation into one of the three pre-defined classes, viz. polite, impolite and neutral, and extracts phrases denoting politeness or impoliteness in that turn simultaneously. As there is no such readily available data, we prepare a conversational dataset, PoDial for mental health counseling and legal aid for crime victims in English for our experiment. Experimental results demonstrate that our proposed approach is effective and achieves 2.04 points improvement on turn identification accuracy and 2.40 points on phrase extraction F1- score on our dataset over baselines.

Chinese satellite frequency and orbit entity relation extraction method based on dynamic integrated learning

Given the scarcity of Satellite Frequency and Orbit (SFO) resources, it holds paramount importance to establish a comprehensive knowledge graph of SFO field (SFO-KG) and employ knowledge reasoning technology to automatically mine available SFO resources. An essential aspect of constructing SFO-KG is the extraction of Chinese entity relations. Unfortunately, there is currently no publicly available Chinese SFO entity Relation Extraction (RE) dataset. Moreover, publicly available SFO text data contain numerous NA (representing for “No Answer”) relation category sentences that resemble other relation sentences and pose challenges in accurate classification, resulting in low recall and precision for the NA relation category in entity RE. Consequently, this issue adversely affects both the accuracy of constructing the knowledge graph and the efficiency of RE processes. To address these challenges, this paper proposes a method for extracting Chinese SFO text entity relations based on dynamic integrated learning. This method includes the construction of a manually annotated Chinese SFO entity RE dataset and a classifier combining features of SFO resource data. The proposed approach combines integrated learning and pre-training models, specifically utilizing Bidirectional Encoder Representation from Transformers (BERT). In addition, it incorporates one-class classification, attention mechanisms, and dynamic feedback mechanisms to improve the performance of the RE model. Experimental results show that the proposed method outperforms the traditional methods in terms of F1 value when extracting entity relations from both balanced and long-tailed datasets.